Ηand dexterities assessment in stroke patients based on augmented reality and machine learning through a box and block test.

A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.

Accuracy of Video-Based Hand Tracking for People With Upper-Body Disabilities.

Utilization of hand-tracking cameras, such as Leap, for hand rehabilitation and functional assessments is an innovative approach to providing affordable alternatives for people with disabilities. However, prior to deploying these commercially-available tools, a thorough evaluation of their performance for disabled populations is necessary. In this study, we provide an in-depth analysis of the accuracy of Leap's hand-tracking feature for both individuals with and without upper-body disabilities for common dynamic tasks used in rehabilitation. Leap is compared against motion capture with conventional techniques such as signal correlations, mean absolute errors, and digit segment length estimation. We also propose the use of dimensionality reduction techniques, such as Principal Component Analysis (PCA), to capture the complex, high-dimensional signal spaces of the hand. We found that Leap's hand-tracking performance did not differ between individuals with and without disabilities, yielding average signal correlations between 0.7-0.9. Both low and high mean absolute errors (between 10-80mm) were observed across participants. Overall, Leap did well with general hand posture tracking, with the largest errors associated with the tracking of the index finger. Leap's hand model was found to be most inaccurate in the proximal digit segment, underestimating digit lengths with errors as high as 18mm. Using PCA to quantify differences between the high-dimensional spaces of Leap and motion capture showed that high correlations between latent space projections were associated with high accuracy in the original signal space. These results point to the potential of low-dimensional representations of complex hand movements to support hand rehabilitation and assessment.

Determinants of hand pulse wave velocity and hand pulse transit time in healthy adults.

Arterial pulse wave velocity (PWV) is recognized as a convenient method to assess peripheral vascular stiffness. This study explored the clinical characteristics of hand PWV (hPWV) and hand pulse transit time (hPTT) in healthy adults (sixty males = 42.4 ± 13.9 yrs; sixty-four females = 42.8 ± 13.9 yrs) voluntarily participated in this study. The arterial pulse waveform and the anatomical distance from the radial styloid process to the tip of the middle finger of both hands were recorded in the sitting position. The hPWV was calculated as the traversed distance divided by hPTT between those two points. Male subjects showed significantly greater hPWV, systolic blood pressure, and pulse pressure than age-matched female subjects, while the hPTT was not significantly different between genders. Multiple linear regression analysis showed that gender is a common determinant of hPWV and hPTT, and that age and heart rate (HR) were negatively correlated with hPWV and hPTT, respectively. We conclude that male subjects have greater hPWV than female subjects. Ageing is associated with decreased hPWV, while increased HR is associated with a smaller hPTT. The hPWV and hPTT might be used as non-invasive indices to characterise the ageing and arterial stiffness of peripheral blood vessels.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals.

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

End-to-End Ultrasonic Hand Gesture Recognition.

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

Developing a Novel Prosthetic Hand with Wireless Wearable Sensor Technology Based on User Perspectives: A Pilot Study.

Myoelectric hands are beneficial tools in the daily activities of people with upper-limb deficiencies. Because traditional myoelectric hands rely on detecting muscle activity in residual limbs, they are not suitable for individuals with short stumps or paralyzed limbs. Therefore, we developed a novel electric prosthetic hand that functions without myoelectricity, utilizing wearable wireless sensor technology for control. As a preliminary evaluation, our prototype hand with wireless button sensors was compared with a conventional myoelectric hand (Ottobock). Ten healthy therapists were enrolled in this study. The hands were fixed to their forearms, myoelectric hand muscle activity sensors were attached to the wrist extensor and flexor muscles, and wireless button sensors for the prostheses were attached to each user's trunk. Clinical evaluations were performed using the Simple Test for Evaluating Hand Function and the Action Research Arm Test. The fatigue degree was evaluated using the modified Borg scale before and after the tests. While no statistically significant differences were observed between the two hands across the tests, the change in the Borg scale was notably smaller for our prosthetic hand (p = 0.045). Compared with the Ottobock hand, the proposed hand prosthesis has potential for widespread applications in people with upper-limb deficiencies.

Anthropomorphic Tendon-Based Hands Controlled by Agonist-Antagonist Corticospinal Neural Network.

This article presents a study on the neurobiological control of voluntary movements for anthropomorphic robotic systems. A corticospinal neural network model has been developed to control joint trajectories in multi-fingered robotic hands. The proposed neural network simulates cortical and spinal areas, as well as the connectivity between them, during the execution of voluntary movements similar to those performed by humans or monkeys. Furthermore, this neural connection allows for the interpretation of functional roles in the motor areas of the brain. The proposed neural control system is tested on the fingers of a robotic hand, which is driven by agonist-antagonist tendons and actuators designed to accurately emulate complex muscular functionality. The experimental results show that the corticospinal controller produces key properties of biological movement control, such as bell-shaped asymmetric velocity profiles and the ability to compensate for disturbances. Movements are dynamically compensated for through sensory feedback. Based on the experimental results, it is concluded that the proposed biologically inspired adaptive neural control system is robust, reliable, and adaptable to robotic platforms with diverse biomechanics and degrees of freedom. The corticospinal network successfully integrates biological concepts with engineering control theory for the generation of functional movement. This research significantly contributes to improving our understanding of neuromotor control in both animals and humans, thus paving the way towards a new frontier in the field of neurobiological control of anthropomorphic robotic systems.

Implicit visuospatial sequence representations are accessible in both the practice and the transfer hand.

A serial reaction time task was used to test whether the representations of a probabilistic second-order sequence structure are (i) stored in an effector-dependent, effector-independent intrinsic or effector-independent visuospatial code and (ii) are inter-manually accessible. Participants were trained either with the dominant or non-dominant hand. Tests were performed with both hands in the practice sequence, a random sequence, and a mirror sequence. Learning did not differ significantly between left and right-hand practice, suggesting symmetric intermanual transfer from the dominant to the non-dominant hand and vice versa. In the posttest, RTs were shorter for the practice sequence than for the random sequence, and longest for the mirror sequence. Participants were unable to freely generate or recognize the practice sequence, indicating implicit knowledge of the probabilistic sequence structure. Because sequence-specific learning did not differ significantly between hands, we conclude that representations of the probabilistic sequence structure are stored in an effector-independent visuospatial code.

Grasping objects with the aid of haptics.

A smart suction cup uses haptics to supplement vision for exploration of objects in a grasping task.

Cortical activations associated with spatial remapping of finger touch using EEG.

The spatial coding of tactile information is functionally essential for touch-based shape perception and motor control. However, the spatiotemporal dynamics of how tactile information is remapped from the somatotopic reference frame in the primary somatosensory cortex to the spatiotopic reference frame remains unclear. This study investigated how hand position in space or posture influences cortical somatosensory processing. Twenty-two healthy subjects received electrical stimulation to the right thumb (D1) or little finger (D5) in three position conditions: palm down on right side of the body (baseline), hand crossing the body midline (effect of position), and palm up (effect of posture). Somatosensory-evoked potentials (SEPs) were recorded using electroencephalography. One early-, two mid-, and two late-latency neurophysiological components were identified for both fingers: P50, P1, N125, P200, and N250. D1 and D5 showed different cortical activation patterns: compared with baseline, the crossing condition showed significant clustering at P1 for D1, and at P50 and N125 for D5; the change in posture showed a significant cluster at N125 for D5. Clusters predominated at centro-parietal electrodes. These results suggest that tactile remapping of fingers after electrical stimulation occurs around 100-125 ms in the parietal cortex.

The Effects of Hand Tremors on the Shooting Performance of Air Pistol Shooters with Different Skill Levels.

Physiologic hand tremors are a critical factor affecting the aim of air pistol shooters. However, the extent of the effect of hand tremors on shooting performance is unclear. In this study, we aim to explore the relationship between hand tremors and shooting performance scores as well as investigate potential links between muscle activation and hand tremors. In this study, 17 male air pistol shooters from China's national team and the Air Pistol Sports Center were divided into two groups: the elite group and the sub-elite group. Each participant completed 40 shots during the experiment, with shooters' hand tremors recorded using three-axis digital accelerometers affixed to their right hands. Muscle activation was recorded using surface electromyography on the right anterior deltoid, posterior deltoid, biceps brachii (short head), triceps brachii (long head), flexor carpi radialis, and extensor carpi radialis. Our analysis revealed weak correlations between shooting scores and hand tremor amplitude in multiple directions (middle-lateral, ML: r2 = -0.22, p < 0.001; vertical, VT: r2 = -0.25, p < 0.001), as well as between shooting scores and hand tremor complexity (ML: r2 = -0.26, p < 0.001; VT: r2 = -0.28, p < 0.001), across all participants. Notably, weak correlations between shooting scores and hand tremor amplitude (ML: r2 = -0.27, p < 0.001; VT: r2 = -0.33, p < 0.001) and complexity (ML: r2 = -0.31, p < 0.001) were observed in the elite group but not in the sub-elite group. Moderate correlation were found between the biceps brachii (short head) RMS and hand tremor amplitude in the VT and ML directions (ML: r2 = 0.49, p = 0.010; VT: r2 = 0.44, p = 0.025) in all shooters, with a moderate correlation in the ML direction in elite shooters (ML: r2 = 0.49, p = 0.034). Our results suggest that hand tremors in air pistol shooters are associated with the skill of the shooters, and muscle activation of the biceps brachii (long head) might be a factor affecting hand tremors. By balancing the agonist and antagonist muscles of the shoulder joint, shooters might potentially reduce hand tremors and improve their shooting scores.

Development of a Two-Finger Haptic Robotic Hand with Novel Stiffness Detection and Impedance Control.

Haptic hands and grippers, designed to enable skillful object manipulation, are pivotal for high-precision interaction with environments. These technologies are particularly vital in fields such as minimally invasive surgery, where they enhance surgical accuracy and tactile feedback: in the development of advanced prosthetic limbs, offering users improved functionality and a more natural sense of touch, and within industrial automation and manufacturing, they contribute to more efficient, safe, and flexible production processes. This paper presents the development of a two-finger robotic hand that employs simple yet precise strategies to manipulate objects without damaging or dropping them. Our innovative approach fused force-sensitive resistor (FSR) sensors with the average current of servomotors to enhance both the speed and accuracy of grasping. Therefore, we aim to create a grasping mechanism that is more dexterous than grippers and less complex than robotic hands. To achieve this goal, we designed a two-finger robotic hand with two degrees of freedom on each finger; an FSR was integrated into each fingertip to enable object categorization and the detection of the initial contact. Subsequently, servomotor currents were monitored continuously to implement impedance control and maintain the grasp of objects in a wide range of stiffness. The proposed hand categorized objects' stiffness upon initial contact and exerted accurate force by fusing FSR and the motor currents. An experimental test was conducted using a Yale-CMU-Berkeley (YCB) object set consisted of a foam ball, an empty soda can, an apple, a glass cup, a plastic cup, and a small milk packet. The robotic hand successfully picked up these objects from a table and sat them down without inflicting any damage or dropping them midway. Our results represent a significant step forward in developing haptic robotic hands with advanced object perception and manipulation capabilities.

Illusory Directional Sensation Induced by Asymmetric Vibrations Influences Sense of Agency and Velocity in Wrist Motions.

Illusory directional sensations are generated through asymmetric vibrations applied to the fingertips and have been utilized to induce upper-limb motions in the rehabilitation and training of patients with visual impairment. However, its effects on motor control remain unclear. This study aimed to verify the effects of illusory directional sensations on wrist motion. We conducted objective and subjective evaluations of wrist motion during a motor task, while inducing an illusory directional sensation that was congruent or incongruent with wrist motion. We found that, when motion and illusory directional sensations were congruent, the sense of agency for motion decreased. This indicates an induction sensation of the hand being moved by the illusion. Interestingly, although no physical force was applied to the hand, the angular velocity of the wrist was higher in the congruent condition than that in the no-stimulation condition. The angular velocity of the wrist and electromyography signals of the agonist muscles were weakly positively correlated, suggesting that the participants may have increased their wrist velocity. In other words, the congruence between the direction of motion and illusory directional sensation induced the sensation of the hand being moved, even though the participants' wrist-motion velocity increased. This phenomenon can be explained by the discrepancy between the sensation of active motion predicted by the efferent copy, and that of actual motion caused by the addition of the illusion. The findings of this study can guide the design of novel rehabilitation methods.

Tactile shape discrimination for moving stimuli.

In this study, we explored spatial-temporal dependencies and their impact on the tactile perception of moving objects. Building on previous research linking visual perception and human movement, we examined if an imputed motion mechanism operates within the tactile modality. We focused on how biological coherence between space and time, characteristic of human movement, influences tactile perception. An experiment was designed wherein participants were stimulated on their right palm with tactile patterns, either ambiguous (incongruent conditions) or non-ambiguous (congruent conditions) relative to a biological motion law (two-thirds power law) and asked to report perceived shape and associated confidence. Our findings reveal that introducing ambiguous tactile patterns (1) significantly diminishes tactile discrimination performance, implying motor features of shape recognition in vision are also observed in the tactile modality, and (2) undermines participants' response confidence, uncovering the accessibility degree of information determining the tactile percept's conscious representation. Analysis based on the Hierarchical Drift Diffusion Model unveiled the sensitivity of the evidence accumulation process to the stimulus's informational ambiguity and provides insight into tactile perception as predictive dynamics for reducing uncertainty. These discoveries deepen our understanding of tactile perception mechanisms and underscore the criticality of predictions in sensory information processing.

Towards humanlike grasp in robotic hands: mechanical implementation of force synergies.

In the field of robotic hands, finger force coordination is usually achieved by complex mechanical structures and control systems. This study presents the design of a novel transmission system inspired from the physiological concept of force synergies, aiming to simplify the control of multifingered robotic hands. To this end, we collected human finger force data during six isometric grasping tasks, and force synergies (i.e. the synergy weightings and the corresponding activation coefficients) were extracted from the concatenated force data to explore their potential for force modulation. We then implemented two force synergies with a cable-driven transmission mechanism consisting of two spring-loaded sliders and five V-shaped bars. Specifically, we used fixed synergy weightings to determine the stiffness of the compression springs, and the displacements of sliders were determined by time-varying activation coefficients. The derived transmission system was then used to drive a five-finger robotic hand named SYN hand. We also designed a motion encoder to selectively activate desired fingers, making it possible for two motors to empower a variety of hand postures. Experiments on the prototype demonstrate successful grasp of a wide range of objects in everyday life, and the finger force distribution of SYN hand can approximate that of human hand during six typical tasks. To our best knowledge, this study shows the first attempt to mechanically implement force synergies for finger force modulation in a robotic hand. In comparison to state-of-the-art robotic hands with similar functionality, the proposed hand can distribute humanlike force ratios on the fingers by simple position control, rather than resorting to additional force sensors or complex control strategies. The outcome of this study may provide alternatives for the design of novel anthropomorphic robotic hands, and thus show application prospects in the field of hand prostheses and exoskeletons.

Human local field potentials in motor and non-motor brain areas encode upcoming movement direction.

Limb movement direction can be inferred from local field potentials in motor cortex during movement execution. Yet, it remains unclear to what extent intended hand movements can be predicted from brain activity recorded during movement planning. Here, we set out to probe the directional-tuning of oscillatory features during motor planning and execution, using a machine learning framework on multi-site local field potentials (LFPs) in humans. We recorded intracranial EEG data from implanted epilepsy patients as they performed a four-direction delayed center-out motor task. Fronto-parietal LFP low-frequency power predicted hand-movement direction during planning while execution was largely mediated by higher frequency power and low-frequency phase in motor areas. By contrast, Phase-Amplitude Coupling showed uniform modulations across directions. Finally, multivariate classification led to an increase in overall decoding accuracy (>80%). The novel insights revealed here extend our understanding of the role of neural oscillations in encoding motor plans.

Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes.

Classification-based myoelectric control has attracted significant interest in recent years, leading to prosthetic hands with advanced functionality, such as multi-grip hands. Thus far, high classification accuracies have been achieved by increasing the number of surface electromyography (sEMG) electrodes or adding other sensing mechanisms. While many prescribed myoelectric hands still adopt two-electrode sEMG systems, detailed studies on signal processing and classification performance are still lacking. In this study, nine able-bodied participants were recruited to perform six typical hand actions, from which sEMG signals from two electrodes were acquired using a Delsys Trigno Research+ acquisition system. Signal processing and machine learning algorithms, specifically, linear discriminant analysis (LDA), k-nearest neighbors (KNN), and support vector machines (SVM), were used to study classification accuracies. Overall classification accuracy of 93 ± 2%, action-specific accuracy of 97 ± 2%, and F1-score of 87 ± 7% were achieved, which are comparable with those reported from multi-electrode systems. The highest accuracies were achieved using SVM algorithm compared to LDA and KNN algorithms. A logarithmic relationship between classification accuracy and number of features was revealed, which plateaued at five features. These comprehensive findings may potentially contribute to signal processing and machine learning strategies for commonly prescribed myoelectric hand systems with two sEMG electrodes to further improve functionality.

E-BabyNet: Enhanced Action Recognition of Infant Reaching in Unconstrained Environments.

Machine vision and artificial intelligence hold promise across healthcare applications. In this paper, we focus on the emerging research direction of infant action recognition, and we specifically consider the task of reaching which is an important developmental milestone. We develop E-babyNet, a lightweight yet effective neural-network-based framework for infant action recognition that leverages the spatial and temporal correlation of bounding boxes of infants' hands and objects to reach for to determine the onset and offset of the reaching action. E-babyNet consists of two main layers based on two LSTM and a Bidirectional LSTM (BiLSTM) model, respectively. The first layer provides a pre-evaluation of the reaching action for each hand by providing onset and offset keyframes. Then, the biLSTM model merges the previous outputs to deliver an outcome of the reaching actions detection for each frame including the reaching hand. We evaluated our approach against four other lightweight structures using a dataset comprising 5,865 annotated images resulting in 16,337 bounding boxes from 375 distinctive infant reaching actions performed while sitting by different subjects in unconstrained (home/clinic) environments. Results illustrate the effectiveness of our approach and ability to provide reliable reaching action detection and offer onset and offset keyframes with a precision of one frame. Moreover, the biLSTM layer can handle the transition between reaching actions and help reduce false detections.

Effects of finger pinch motor imagery on short-latency afferent inhibition and corticospinal excitability.

Motor imagery is a cognitive process involving the simulation of motor actions without actual movements. Despite the reported positive effects of motor imagery training on motor function, the underlying neurophysiological mechanisms have yet to be fully elucidated. Therefore, the purpose of the present study was to investigate how sustained tonic finger-pinching motor imagery modulates sensorimotor integration and corticospinal excitability using short-latency afferent inhibition (SAI) and single-pulse transcranial magnetic stimulation (TMS) assessments, respectively. Able-bodied individuals participated in the study and assessments were conducted under two experimental conditions in a randomized order between participants: (1) participants performed motor imagery of a pinch task while observing a visual image displayed on a monitor (Motor Imagery), and (2) participants remained at rest with their eyes fixed on the monitor displaying a cross mark (Control). For each condition, sensorimotor integration and corticospinal excitability were evaluated during sustained tonic motor imagery in separate sessions. Sensorimotor integration was assessed by SAI responses, representing inhibition of motor-evoked potentials (MEPs) in the first dorsal interosseous muscle elicited by TMS following median nerve stimulation. Corticospinal excitability was assessed by MEP responses elicited by single-pulse TMS. There was no significant difference in the magnitude of SAI responses between motor imagery and Control conditions, while MEP responses were significantly facilitated during the Motor Imagery condition compared to the Control condition. These findings suggest that motor imagery facilitates corticospinal excitability, without altering sensorimotor integration, possibly due to insufficient activation of the somatosensory circuits or lack of afferent feedback during sustained tonic motor imagery.